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ABSTRACT 



Information on effective policies/practices for assessing 
student performance in Texas ' tech prep programs was gathered and 
disseminated during a project involving the following activities: formation 
of an advisory committee to oversee the data collection/analysis and product 
development processes; compilation/review of performance assessment documents 
from tech prep consortia throughout Texas; administration of the 1997 Tech 
Prep Student Outcomes Assessment Questionnaire (1,000 copies of which were 
mailed to Texas's 25 tech prep directors); analysis of the 115 usable 
questionnaires eventually returned (response rate, 11.5%); and phone 
interviews with 27 individuals from a sample of Texas' tech prep directors 
and others considered knowledgeable about student performance assessment . It 
was discovered that Texas tech prep practitioners are using a variety of 
methods to assess their students' competencies in vocational -technical 
subject matter, academic subject matter, employability skills, and workplace 
skills. Performance assessment was conducted most frequently at the course 
(rather than program) level. (Appendixes constituting approximately 60% of 
this document contain the following: list of project advisory council 
members; minutes of advisory council meeting; phone interview protocol; 1997 
tech prep student outcome assessment questionnaire; survey- related 
correspondence to consortia directors; document analysis rating system; and 
frequencies for the questionnaire.) (MN) 
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Executive Summary 



This is an executive summary of The Final Report on Effective Tech Prep Policies and 
Practices: Performance Assessment, a project funded by Carl Perkins funds distributed 
by the Texas Higher Education Coordinating Board. The purpose of the project was to 
gather and disseminate information on effective policies and practices for assessing the 
performance of students in Tech Prep programs in Texas. The project was conducted 
by the Strategic Planning, Evaluation of Curriculum, and Assessment of Performance 
(SPECAP) Research Group at Texas Tech University. The project researchers utilized 
document analysis, surveys, and phone interviews to gather both quantitative and 
qualitative data. A brief description of the products that emerged from the project are 
discussed in this executive summary, along with a brief summary of the conclusions 
and policy recommendations. 

Products 

Performance assessment documents were reviewed for possible inclusion in The 
Tech Prep Handbook: Performance Assessment. The Tech Prep Handbook: 
Performance Assessment has been designed so that Tech Prep practitioners involved 
in the assessment of student performance have a ready source of models that they can 
adapt in assessing their own students' performance. 

The Final Report on Effective Tech Prep Policies and Practices: Performance 
Assessment is another product created by SPECAP project researchers. In The Final 



Report, project researchers discuss in detail the activities of the SPECAP Research 
Group "the document analysis, phone interviews, and surveys, as well as the products 
created -- the handbook, final report, and conference presentations. The Final Report 
also contains the conclusions and policy recommendations of the SPECAP researchers 
based on their analyses of the data gathered for the project. These conclusions and 
recommendations are presented in the section that follows. 

Conclusions and Policy Recommendations 

A number of conclusions are discussed in The Final Report on Effective Tech 
Prep Policies and Practices: Performance Assessment. Upon analyses of the data 
gathered through the document collection process, interviews, and surveys, the 
SPECAP researchers arrived at the following conclusions: 

• Tech Prep practitioners are utilizing a variety of methods to 
assess the performance of Tech Prep students. 

• Tech Prep student performance measures include competencies 
in vocational/ technical subject matter, academic subject matter, 
employability skills, and workplace skills. 



• Tech Prep student performance assessment is most frequently 
conducted at the level of the course, rather than at the program 
level. 

A number of policy recommendations are also discussed in the Final Report. 
Based on the findings of the project, the following policy recommendations are 
suggested; 



• Assess student performance at the level of the Tech Prep 
program, measuring both cognitive and affective outcomes. 

• Create longitudinal student tracking systems to track program- 
specific outcomes, such as student success rate on licensure 
exams, and student satisfaction. 

• Create a statewide longitudinal tracking system to track 
consistently such Tech Prep student outcomes as graduation, 
transfer, and job placement rates. 

• Involve business and industry experts in verifying mastery of 
employability and job competency skills in work-based 
components of Tech Prep programs. 



Introduction 



This final report will describe the activities and the findings of the Strategic 
Planning, Evaluation of Curriculum, and Assessment of Performance (SPECAP) 
Research Group at Texas Tech University. The SPECAP Research Group was 
awarded a Carl Perkins grant in 1 996 entitled "Effective Tech Prep Policies and 
Practices: Performance Assessment." The purpose of the grant project was to gather 
and disseminate information on effective policies and practices in the assessment of 
Tech Prep student performance in Texas. The grant project was designed as a 
continuation of the previous year's efforts by the SPECAP Research Group to identify 
and disseminate information on effective Tech Prep policies and practices in curriculum 
development and evaluation in Texas. Many of the models, processes, and products 
used to examine curriculum development and evaluation in 1995 were modified and 
refined to examine student performance assessment activities in this year's grant 
activities. 

The model that the SPECAP researchers used to examine the assessment of 
student performance is shown in Figure 1. The SPECAP Performance Assessment 
Planning Model is an adaptation of the SPECAP Strategic Planning Model used in the 
1994 grant to examine the strategic planning process in Tech Prep consortia (Figure 2). 
The SPECAP Performance Assessment Planning Model has five components: 
positioning the architects, development of the plan, implementation of the plan, 
evaluation of the plan, and improvement of the plan. The model was validated by the 



6 



8 



Tech Prep experts on our SPECAP advisory council as a useful and valid description of 
the processes used to develop a student performance assessment plan. 



Figure 1. SPECAP Performance Assessment Planning Model. 

The major activities conducted by the SPECAP Research Group during the 
course of this year's grant project will be described in detail in the sections that follow. 
These activities include: advisory council meetings, document analysis, survey 
activities, phone interviews, conference presentations, handbook, and final report. 
Following this description of activities, the final sections of the report focus on 
conclusions and policy recommendations. 
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Figure 2. SPECAP Strategic Planning Model. 



Advisory Council Meetings 

An advisory council was formed to provide feedback to the SPECAP Research 
Group on grant activities and products. Appendix A lists the experts who agreed to 
serve on this advisory council. In selecting the advisory council members, an effort was 
made to choose individuals who were both knowledgeable about Tech Prep in Texas, 
and who worked with Tech Prep programs in different areas of the state. The advisory 
council members were chosen based on nominations from representatives of the Texas 
Higher Education Coordinating Board. 

There was one advisory council meeting held in conjunction with the Texas State 
Tech Prep Conference in Austin in March of 1997. Appendix B contains the minutes 
describing the discussion that took place at this meeting. The primary purpose of this 
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advisory council meeting was to discuss preliminary findings from phone interviews, 
questionnaires, and documents used to gather data on the performance assessment 
planning process in Tech Prep programs. Prior to this meeting, the advisory council 
members had been contacted by phone on several occasions to request feedback on 
the contents of the phone interview protocol and questionnaire, found in Appendices C 
and D respectively. The advisory council members were also requested to provide 
feedback on the rating system used to rate the performance assessment documents to 
be included in the handbook. A final task of the advisory council was to provide 
feedback on the project products -- final report and the handbook, produced and 
disseminated by the SPECAP Research Group. These products were revised to 
incorporate changes based on the council members' recommendations. 

Document Collection and Analysis Process 

To more fully understand the range of Tech Prep performance assessment 
policies and practices, SPECAP researchers compiled Tech Prep performance 
assessment documents from around the state. To obtain these documents, SPECAP 
researchers sent letters to each Tech Prep consortium director asking for any written 
materials that they had describing the assessment of Tech Prep student performance. 
A copy of the letter that was sent to consortium directors requesting these performance 
assessment documents can be found in Appendix E. Documents describing 
performance assessment policies and practices were also requested by SPECAP 
researchers from the individuals selected for phone interviews. 
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All performance assessment documents received were then rated using the 
rating system found in Appendix F. The purpose of this rating system was to select for 
inclusion in the handbook those documents most relevant to, and representative of, 
performance assessment policies and practices. Each performance assessment 
document was scored by three different SPECAP researchers, and the average score 
from these three ratings was used to determine which documents would be included in 
the handbook. 

In categorizing these documents, SPECAP researchers examined all the 
documents within broad categories of performance assessment. For example, 
performance assessment documents categorized as student competency profiles were 
rated as a group to choose the highest scoring example within this category for 
inclusion in the handbook. The same process was repeated for each of the other broad 
categories of performance assessment documents - student surveys, employer 
surveys, and student portfolios. 

1997 Tech Prep Student Outcomes Assessment Questionnaire 

A survey was utilized to gather information from experts throughout the state on 
performance assessment policies and practices. The SPECAP Performance 
Assessment Model was used as the conceptual framework in designing the 1997 Tech 
Prep Student Outcomes Assessment Questionnaire. In addition to questions covering 
the five sections of this model, some basic demographic questions about the 



respondents were also included. The final version of the 1997 Tech Prep Student 
Outcomes Assessment Questionnaire can be found in Appendix D. 

In designing the questionnaire, SPECAP researchers created a pilot draft of the 
questionnaire for review by the advisory council. Based on their suggestions for 
revisions, ambiguous questions on the questionnaire were either revised or eliminated. 
The final version of the questionnaire was two pages long, with seventeen closed- 
ended questions regarding the planning for student performance assessment. To 
simplify data entry, the questionnaires responses were directly entered into a data file 
for analysis by a microcomputer statistical package. 

In deciding on the sampling design for this questionnaire, the SPECAP 
researchers relied heavily on the advice of the advisory council members on how best 
to reach the performance assessment experts in their consortia. The advisory council 
recommended that each consortium director receive questionnaires proportional to the 
number of students they had enrolled in Tech Prep programs within their consortia. 
Based on this recommendation, the sampling was designed so that a proportional 
number of questionnaires was sent to each consortium director based on the most 
recent Tech Prep student enrollment figures obtained from the Texas Higher Education 
Coordinating Board. Appendix G displays these enrollment figures and the number of 
questionnaires that were sent to and returned from individuals within each of the 
twenty-five consortia. 

Another recommendation made by the advisory council members was that the 
Tech Prep directors should decide who within their respective consortia should receive 
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the questionnaires. Advisory council members stated that the Tech Prep consortium 
directors would be the individuals most knowledgeable about who within their 
consortium had been involved in the development and evaluation of the student 
performance assessment plan. As a result of this recommendation, SPECAP 
researchers sent questionnaires directly to the Tech Prep directors, with a cover letter 
indicating that they were to choose the performance assessment experts within their 
consortia to receive the questionnaires. Appendix H displays the cover letter sent to 
consortia directors with these instructions. The directors were asked to send 
questionnaires to all stakeholders within their consortia who were involved in developing 
student performance assessment plans. 

A total of 1 ,000 questionnaires were mailed to the 25 Tech Prep directors around 
the state for further distribution to the performance assessment experts within their 
consortia. A total of 1 15 usable questionnaires were received by the SPECAP 
researchers by the end of April of 1997. Since the SPECAP researchers have no way 
of knowing how many of the 1 ,000 questionnaires were actually mailed out by 
consortium directors, it is not possible to calculate an overall response rate for the 
survey. The questionnaires received were keypunched as they were received, and all 
usable questionnaires received were entered and analyzed. The results of that analysis 
will be presented in conjunction with the findings from the phone interviews, which will 
be described in the section that follows. 
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Phone Interviews 

To more fully understand performance assessment policies and practices, a 
series of phone interviews were conducted by SPECAP researchers. A phone interview 
protocol was developed using the SPECAP Performance Assessment Model as the 
conceptual framework. The questions were designed to gather information about how 
student performance assessment plans were developed and evaluated. The phone 
interview data were also designed to complement the quantitative data gathered with 
the 1997 Tech Prep Student Outcomes Assessment Questionnaire. Feedback on a 
draft of the phone interview protocol was received from our advisory council members 
prior to pilot testing the instrument. Revisions suggested by advisory council members, 
along with those suggested by the individual chosen to pilot test the instrument, were 
incorporated into the final phone interview protocol, which can be found in Appendix C. 
The advisory council members also suggested that Tech Prep directors nominate the 
individuals most knowledgeable about performance assessment within their consortia to 
be included in the phone interview sample. Each of the twenty-five Tech Prep directors 
was included in the sample of individuals to be interviewed, along with individuals that 
they nominated from their consortia who they considered to be knowledgeable about 
student performance assessment. The phone interviews were conducted by three 
SPECAP researchers over a two-month period from February through March of 1997. 

A total of 27 interviews were completed, with each interview taking approximately thirty 
minutes to conduct. Each individual interviewed was promised confidentiality in the 
dissemination of the findings. At the completion of each interview, the SPECAP 
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researchers typed up their interview notes and sent a thank you card to the individual 
that they had interviewed. 



Upon completion of all the phone interviews, the interview notes from all of the 
Interviews were combined, and the data coded for analysis. The data gathered from 
the phone interviews helped SPECAP researchers more fully understand and explain 
the findings gathered with the survey instrument. Since the SPECAP researchers 
promised confidentiality to the individuals interviewed, quotes used in the findings that 
follow do not provide information about the names of individuals interviewed, nor 
identifying information about the consortia with which they are affiliated. The findings of 
the phone interviews are presented in conjunction with the findings from the survey so 
that the reader has a more complete understanding of performance assessment 
policies and practices in Tech Prep programs in Texas. 



The findings from the 1997 Tech Prep Student Outcomes Assessment 
Questionnaire and the phone interviews have been integrated for the purposes of this 
final report. The findings will be discussed in the following sections: response 
characteristics, involvement in student outcomes assessment, the assessment process, 
developing/implementing the student assessment plan, and evaluating/improving the 
student outcomes assessment plan. The frequencies for all the questions on the 1997 
Tech Prep Student Outcomes Assessment Questionnaire can be found in Appendix I. 




Findings from the Survey and Phone Interviews 
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Response Characteristics 



Questionnaires were received from 19 out of 25 of the Tech Prep consortia, 
indicating that there is geographic representation in the responses (See Appendix J). 
The number of responses varied considerably by consortia, ranging from a low of one 
to a high of 16, with an average of just over 6 responses per consortium. There were 
six consortia from which no responses were received. To ensure responses from all 
Tech Prep consortia, future surveys may require the joint sponsorship of the Texas 
Higher Education Coordinating Board and the SPECAP research group. 

The survey respondents represented performance assessment experts from a 
wide diversity of career pathways. 




Business Industrial/ Computer 
Trade Systems 



Figure 3. Career pathways of respondents to the questionnaire. 



Slightly over two-thirds of respondents are affiliated with a business career pathway, 
with two-fifths being affiliated with an industrial/trade pathway, and slightly more than 
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one-third with a computer systems career. This distribution of responses indicates that 
the consortium directors did target questionnaires towards a wide variety of individuals 
within their consortia. The fact that the percentages do not add up to 100% indicates 
that a number of respondents, such as Tech Prep directors, are affiliated with multiple 
career pathways. 

The Tech Prep directors were instructed to target the questionnaires they 
received to as many different stakeholder groups as possible. Figure 4 indicates 
representation from a number of different stakeholder groups in the survey responses. 




Figure 4. Respondents' stakeholder group affiliation. 



The majority of the survey respondents represent either high school faculty or 
administrators, with a much smaller number of community college faculty and 




administrators also represented among the survey respondents. The high 

representation of individuals from the education sector is perhaps not surprising on a 

questionnaire dealing with student performance assessment. Clearly, educators are in 

the best position to assess the school-based experiences of Tech Prep students. Less 

than one-tenth of the respondents come from business or industry. This percentage is 

low, considering that business and industry representatives are often in the best 

position to assess the work-based performance of Tech Prep students. 

One respondent provides additional insights about how business and industry 

representatives participate in student performance assessment: 

The employer evaluates the student on 16 or so competencies after the first six 
weeks of training. The plans are developed jointly with the employer. The 
employer selects the competencies the student should know.. .The competencies 
are built by the employer-it is the curriculum for the student at the workplace. It 
takes about 40 minutes with the employer to develop the competencies . . . For 
the training plans we use the national standards established for high tech 
manufacture as well as the SCANS skills. We go through each section and take 
about 30 to develop with the student at that work station. 



Involvement in Student Outcomes Assessment 



One of the questions on the 1997 Tech Prep Student Outcomes Assessment 
Questionnaire asks respondents "Have you participated in any of the following student 



outcome assessment activities? 
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Figure 5. Top four student assessment activities in which respondents' 
participated. 

Slightly less than two-thirds of the survey respondents (62.6%) indicate that they have 
attended a professional development workshop on student outcomes assessment. 
This suggests that professional development workshops on student outcomes 
assessment are available to Tech Prep practitioners. A much smaller number of 
respondents (30.4%) indicate that they have served on a student outcomes 
assessment committee, or have enrolled in a class on student outcomes assessment 
(1 1 .3%). These relatively small percentages suggest that workshops on student 
outcome assessment are much more prevalent than are these other types of 
professional development/involvement. 



18 






A second question asks respondents to indicate the role(s) that they play in 



developing their student outcome assessment plan. 




Figure 6. The role(s) respondents' play in developing student outcome 
assessment plans. 

The role most commonly indicated (52.2%) is that of vocational/technical subject matter 
expert, no doubt reflecting the fact that the majority of respondents are 
vocational/technical faculty. Other responses give additional insights into the types of 
individuals who responded to the survey, with roughly a quarter indicating that they 
played roles as experts in workplace competencies (27.8%), academic subject matter 
(24.3%), or as resource providers (24.3%). These roles correspond with those one 
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might expect from representatives from business and industry, academic subject matter 
faculty, and administrators, respectively. 

The Assessment Process 

A number of questions on the survey elicit information on the general processes 
involved in developing, implementing, and evaluating the student outcomes assessment 
plan. One of these questions asks respondents how long it took them to develop, 
implement, and evaluate their student outcome assessment plan. The average number 
of months reported is a little over nine months, or roughly the equivalent of one 
academic year. Respondents are also asked what percentage of their total assigned 
work load did they devote to developing, implementing, and evaluating their student 
outcomes assessment plan. The average percentage of one's total time devoted to the 
assessment process is reported to be slightly under one-fifth (17.9%). The time 
required to develop, implement, and evaluate their student outcomes assessment plan 
is clearly a significant component of the total time that respondents indicate they work. 

Respondents also indicated what groups have been involved in developing, 
implementing implementing, and evaluating their student outcome assessment plans. 
The groups most often reported as involved in this process include high school faculty 
(72.6%), community college faculty (52.2%), high school administrators (48.7%), 
business/industry representatives (42.5%), and community college administrators 





Figure 7. Groups involved in developing, implementing, and evaluating student 
outcome assessment plans. 



It is important to note that more than two-fifths of these student outcomes assessment 
plans involve business/industry representatives in their development, implementation, 
and evaluation. Clearly, involvement of business/industry experts is seen as essential 
by many involved in Tech Prep programs in assessing student performance. 

A final set of questions asks respondents to indicate their agreement with 
statements regarding their overall assessment planning process. 
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Figure 8. Respondents who "agreed" and "strongly agreed" to statements 
regarding their overall assessment process. 

A large majority of the respondents "agreed" or "strongly agreed" that adequate time 
and resources were provided (86.2%); that adequate staff development was provided 
(70.7%); and that all essential stakeholder groups were involved (72.1%) in the 
assessment planning process. Clearly, the majority of respondents feel that they have 
had adequate resources, time, training, and involvement of essential stakeholders in 
planning their assessment process. This strong agreement on the adequacy of 
resources and personnel suggests that documenting student performance has been 
given a high priority by Tech Prep consortia and educational institutions. 



Developing/lmplementinq the Student Assessment Plan 



Respondents were asked a number of questions about the development and 
implementation process for their student assessment plan. 




Figure 9. Respondents indicating they "agreed" and "strongly agreed" to 

statements regarding the development and implementation of student 
assessment plans. 



One of the first questions is whether their assessment plan is an entirely new plan, or 
one that has been modified from an existing plan. The majority (58.9%) indicate that 
their assessment plan is a modification of an existing plan. This finding corresponds 
with the findings of last year's survey on curriculum development and evaluation, which 



found a similar percentage reporting that their Tech Prep program curriculum is a 
modification of an existing curriculum. 

A second question asks respondents what levels have been incorporated within 
their assessment plan. 




Q. 



High Employer Comm. 
School College 



Figure 10. Top three levels incorporated into assessment plans. 

The majority (77.0%) indicate that their student assessment plan includes outcomes at 
the high school level (77.0%) and at the two-year college level (53.1%), with a minority 
of respondents also indicating that their plans incorporate outcomes from employers 
(20.4%) and from four-year colleges (8.0%). Clearly, the relatively small number of 
respondents indicating that employer outcomes are included in their student 
assessment plans is a cause for some concern. Finding ways to encourage more Tech 

24 

2B 



Prep programs to include employer assessments of students' workplace competencies 
and employability skills would certainly strengthen student performance assessment 
plans. 

Respondents were also asked to indicate what types of student outcomes 
assessment they presently utilize. 




Figure 1 1 . Types of student outcomes assessment presently being used. 



The majority indicate that they assess competency in vocational/technical subject 
matter (65.5%), as well as in academic subject matter (54.9%). A minority of 
respondents indicate that they assess SCANS competencies (41 .6%), job competency 
(38.9%), or that they use state licensure exams (30.1%) for student outcome 
assessment. Clearly, student assessment plans would be strengthened if greater 
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emphasis is placed on assessing the job and SCANS competencies of Tech Prep 
students. 

A final question asks respondents to indicate what student outcome modalities 
they presently are utilizing. 




Figure 12. Modalities of student outcome assessments presently being used. 

The majority of respondents indicate that they use end-of-course examinations (58.4%) 
as a modality for assessing student outcomes. A minority of respondents indicate that 
they use a number of other modalities, including; student competency profiles (37.2%); 
end-of-program exams (30.1%); student portfolios (26.5%); job competency 
examinations (23.9%); job placement tracking systems (23.9%); and student tracking 
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systems (23.0%). Clearly, the final exam is the most commonly utilized modality for 
assessing student performance. 

One phone respondent discusses in more detail the challenges associated with 
using competency profiles. 



Getting independent school districts to send competency transcripts on time and 
filled out correctly is a major challenge. We have developed individual profiles 
on each course and for each student. When profiles are incomplete, we have to 
track down the teachers to have them complete them. Teachers are sometimes 
unwilling to complete the profiles, because they have to evaluate every student 
that comes to college. 



Another respondent describes other challenges associated with the use of competency 
profiles. 



One of the biggest challenges is helping teachers learn how to document 
competencies. The documentation is lengthy, particularly in business careers 
with upwards to 75 competencies. Teachers need to document quality, because 
offering articulated credit for work done in high school puts the reputation of the 
high school teachers on the line. 



Another respondent provides more detail about how they use portfolios in her 
consortium to assess student outcomes. 



Teachers give projects to be completed on the computer. Students work in 
teams, and are assessed by three different teachers. The portfolios are kept in a 
file system, and handed over to the student's next teacher.. .It gives students an 
idea of where they are at the moment, their capabilities, and allows them to set 
goals. The portfolio incorporates academic, vocational, electives, career 
interests, goals, talents, everything. ..After the students got into what the 
portfolios were doing and saw that they could use them to get jobs and 
scholarships, then it wasn't a problem when they would have to re-do them, 
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perhaps a dozen times, before they were in a final form. ..It gives the student 
focus and self-esteem. 



A Tech Prep director in another consortium offered her belief that portfolios of student 
performance would be used more often in the future. 



Portfolios are statements of what the student can do. They are very 
individualized and can be focused toward the employer if the student goes from 
secondary directly to work or toward another direction should the student want to 
continue with his or her education at the postsecondary level. They speak louder 
than resumes. 



In at least one consortium, a respondent discusses how community college faculty were 
involved in helping high schools assess Tech Prep student performance. 



Community college faculty are in the process of designing an exit test for high 
school seniors to determine which students should get articulated credit from 
their high schools. 



In yet another consortium, a Tech Prep director discusses the benefits of using end-of- 
course exams. 



Using the end-of-course exam has two benefits: it provides consistency at the 
secondary and postsecondary level--so teachers are really teaching toward the 
same thing in order that students can be more successful when they go into the 
second level of the course, and the Coordinating Board has identified end-of- 
course exams as a means for proving requirements have been met to the senior 
colleges. 



Evaluatinq/Improvinq the Student Outcomes Assessment Plan 



A final series of questions asks respondents about their processes for evaluating 
and improving their student outcomes assessment plan. The first question is how often 
their student outcomes assessment plan is evaluated for effectiveness. 




"As “Every 

Needed" Year" 



Figure 13. Top two time intervals used in evaluating assessment plans. 



The majority of respondents indicate that they evaluate their plan “as needed" (46.0%); 
or “every year" (38.3%). This suggests that the majority of respondents closely monitor 
their student outcomes assessment plans for effectiveness. 

A second series of questions asks respondents to indicate their agreement with 
statements about their student outcomes assessment plan. 
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Percentage that 
"Agree" or 
"Strongly Agree" 



Figure 14. Respondents who "agreed" and "strongly agreed" to statements 
regarding their student assessment plans. 



The majority of respondents "agree" or "strongly agree" that their plan captures the 
multiple talents that they are trying to develop in students (83.4%). A majority of 
respondents also "agree" or "strongly agree" that their student outcomes assessment 
plan provides meaningful feedback to faculty (83.1%); students (77.1%); and employers 
(65.6%). Clearly, respondents view their student outcome assessment plans favorably 
as providing meaningful feedback to relevant stakeholder groups and for being 
multidimensional in the outcomes assessed. 




A final series of questions asks respondents to indicate their agreements with 
statements about the importance of student outcomes assessment and the impact of 
Tech Prep programs. 
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Figure 15. Respondents who "agreed" and "strongly agreed" to statements 
regarding the impact of Tech Prep programs. 

A majority of respondents (90.7%) "agree" or "strongly agree" that documenting student 
outcomes is essential to Tech Prep. This widespread agreement that documenting 
student outcomes is essential to Tech Prep reflects a strong sentiment among 
respondents that those involved in Tech Prep have a responsibility to demonstrate the 
positive impact of their programs on students. An even greater percentage of 



respondents “agree" or "strongly agree" that Tech Prep programs have had a positive 
impact on student preparation for further education (94.4%) and on student preparation 
for work (94.4%). 

One respondent expressed the positive impact of Tech Prep programs in the 
following manner: 



Tech Prep programs help students make decisions about careers, allow them to 
explore some things, provide mentors in business. Students like classes in 
which theory is connected to application. Tech Prep programs help students to 
focus on finding employment and help them raise their self-esteem about their 
ability to handle college. 



Another respondent reinforces this theme about the positive influence of Tech Prep 
programs. 



Tech Prep students are encouraged to go on to the community college and the 
four-year college. They can use their skills to earn money while in college. Tech 
Prep helps teach them how to work in teams, as well as academic and workforce 
skills. 



Yet another respondent also emphasizes the positive benefits of Tech Prep programs 
on students: 



Tech Prep students can assess their own life goals, and assess realistically how 
much money they would like to make. They come to understand that they will 
need to delay gratification to be able to reach the salary they want. Tech Prep 
students are motivated to plan, and to engage in worksite learning while in high 
school, allowing them to make money that they can save for college. Students 
who are involved in worksite learning learn the need for academic classes like 
physics, and the importance of a neat attire. 



Another respondent indicates that teachers have also received positive benefits from 
participation in Tech Prep. 

Tech Prep programs have a positive impact on teachers' exposure to the 
workplace. Teachers have a greater awareness of the workplace, and can better 
match their academic content to the real world. They can change their teaching 
strategies to better meet student needs. 

Clearly, the respondents to this survey feel strongly that Tech Programs are having a 
positive impact on student preparation for further education and work. 

Handbook 

One of the purposes of the handbook is to create a sourcebook of sample 
performance assessment documents that Tech Prep practitioners can use as models in 
assessing the performance of their students. The section of the handbook with 
performance assessment documents has been organized so as to provide at least one 
representative sample for each of the broad categories of performance assessment 
documents that have been identified. 

These representative samples of performance assessment documents have 
been included in the handbook so that practitioners involved in developing and 
evaluating student assessment have a ready source of models that they can adapt for 
their own use. The handbook provides a listing of all the documents included by 
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SPECAP researchers on performance assessment, as well as information about how to 
obtain a copy of the document from the source. 

A second purpose of the handbook is to disseminate the results of the 1997 
Tech Prep Student Outcomes Assessment Questionnaire and the phone interviews to 
practitioners involved in assessing student performance. The data gathered from the 
survey provides overview information from a large number of respondents about how 
student assessment plans have been developed and evaluated around the state of 
Texas. The data gathered with the phone interviews provide more detailed information 
from a smaller number of respondents about exactly how student assessment plans 
have been developed and evaluated. Together, the survey and phone interview data 
provide insights which can be used by Tech Prep practitioners in developing and 
evaluating their own student assessment plans. 

A total of four copies of the handbook have been mailed to each of the Tech 
Prep consortium directors for dissemination to individuals and groups involved with the 
assessment of student performance. Copies of the handbook have also been mailed to 
state agency representatives involved in overseeing the assessment of Tech Prep 
student performance. An electronic copy has been mailed to Tech Lynx, for inclusion in 
their state clearinghouse on Tech Prep materials, and a hard copy has been mailed to 
the ERIC Clearinghouse on Adult and Vocational Education, for inclusion in the federal 
ERIC system. 



Final Report 



The final report is designed to describe all the activities of the SPECAP 
Research Group over the 1996-97 grant year. The final report describes: the 
conceptual framework upon which this year’s grant activities is based, the activities of 
the SPECAP Advisory Council, and the conduct of the document analysis, phone 
interviews, survey, and conference presentations. Also included in the final report are 
descriptions of the products created by the SPECAP Research Group - the handbook 
and the final report. The final report uses the findings from the data gathered this grant 
year to draw inferences about implications for policy and practice, and to make 
recommendations for improving the assessment of Tech Prep student performance in 
Texas. 

Five copies of the final report have been distributed to representatives of the 
Texas Higher Education Coordinating Board to meet the reporting requirements for all 
Perkins grant recipients. An electronic copy of the final report has been sent to Tech 
Lynx for inclusion in the state clearinghouse on Tech Prep in Texas, and a copy has 
been sent to the ERIC Clearinghouse on Adult and Vocational Education for inclusion in 
the ERIC system. 



other Means of Dissemination 



Another method of disseminating project findings includes making presentations 
at state and national conferences. The section that follows discusses these conference 
presentations that the SPECAP researchers used to disseminate information. 

Conference Presentations 

The SPECAP Research Group presented their findings at two major 
conferences: the National Tech Prep Network Conference in San Antonio, Texas in 
September of 1996, and the Texas State Tech Prep Conference in Austin, Texas, in 
March of 1997. The presentation for both conferences focused on describing findings 
from last year's grant activities on curriculum development and evaluation used by Tech 
Prep consortia in Texas. A PowerPoint slide presentation was prepared for the 
purposes of these conference presentations. These presentations helped to 
disseminate the findings and activities of the SPECAP Research Group to a wider state 
and national audience. 



Recommendations and Conclusions 

The data gathered by the SPECAP Research Group on student performance 
assessment demonstrate that Tech Prep educators are using a variety of methods to 
document the effectiveness of Tech Prep. The primary methods for documenting Tech 



Prep student outcomes involve testing students at the end of each course, competency 
profiles, portfolios, and assessments of SCANS and job competencies. Additionally, in 
fields like nursing, state licensure examinations are an additional form of assessment of 
student outcomes. 

Other efforts to document student outcomes include surveys of students, 
employers, faculty, and other consortium stakeholders, statewide tests such as the 
TAAS and the TASP, statewide evaluation, and longitudinal student tracking systems. 
Clearly, there are a number of significant efforts to document Tech Prep student 
outcomes at the level of the class, program, consortium, and statewide. What appears 
to be less common are efforts to measure the impact of Tech Prep programs using 
pretests and posttests to determine the "talent" that has been developed by entire Tech 
Prep programs. Rather than attempting to assess the impact of Tech Prep programs 
on students, the majority of assessment activities are class-based, using final exams, 
competency profiles, and portfolios to document that Tech Prep students have 
mastered the subject matter, employability, and job competencies covered within a 
specific class. Relatively few efforts were found to document Tech Prep program 
effects, other than in fields like nursing, where students are required to take state 
licensure examinations. Clearly, Tech Prep student performance assessment would be 
strengthened if more efforts were placed on documenting the impact of Tech Prep 
programs on the development of student talent. 

A second approach to strengthening the assessment of Tech Prep student 
performance is to further involve members of business and industry in the planning for 



37 



39 



student performance assessment. As Tech Prep programs continue to include more 
work-based learning components as integral parts of their program, the need for 
business and industry experts to become more involved in helping plan for assessing 
student performance will become even more essential. Business and industry 
representatives are presently involved in validating Tech Prep curricula, and in 
responding to surveys regarding their satisfaction with Tech Prep student preparation 
for the workplace. To further strengthen student performance assessment plans, 
business and industry representatives will need to become as involved as school-based 
faculty in ensuring that Tech Prep students have mastered job competencies and 
developed employability skills. 

A third approach to strengthening the assessment of Tech Prep student 
performance is to create longitudinal student tracking systems. Some efforts at 
accomplishing this are being undertaken within individual colleges, like at Victoria 
College, and also at the state level, using wage records to track the placement of Tech 
Prep students into the work place. There needs to be a concerted effort to track Tech 
Prep students from high school, into two- or four-year colleges, and then into the work 
force. This will require a coordinated effort between the Tri-Agency staff of the Texas 
Department of Education, the Texas Higher Education Coordinating Board, and the 
Texas Department of Labor. A number of respondents to the phone interviews echoed 
this theme about the need for state assistance. 

I think we need to inquire into some sort of partnership or contracting-see what 

other agencies are collecting data like TEC and tap into or cross reference data. 

Part of the problem is the consistency of the data gathering systems... There is a 
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problem with limited funds available and partnerships could help alleviate that 
problem. 



Another respondent made a similar point in talking about the need for a student tracking 
system. 

We need a statewide system with the information sent to the districts. We don't 
have the personnel to do that ourselves. 

A third respondent reinforces the point about the need for help in collecting student 
data. 



The consortia need to see who they can partner with to streamline the 
information and data gathering. With only three to five employees in a Tech 
Prep consortium, we can't collect all this data without a lot of partnering going on. 



Another respondent also emphasized the need for the state to help in collecting data. 



It is important to find out what impact Tech Prep has on students, particularly to 
do comparisons between Tech Prep and non Tech-Prep students. There should 
be long-term studies of Tech Prep students done by the state. The problem is 
that PEIMS will not add additional fields, because Tech Prep does not have 
enough clout. 



A similar sentiment for state involvement in data collection was expressed by a Tech 
Prep director in a major urban area. 



The state needs to decide what they expect Tech Prep student outcomes to be 
and develop a system to gather this data. There are mechanisms in place for 



schools to report data to the Coordinating Board. The fundamental question is 
deciding what to measure. 



Another Tech Prep director from a major urban area in the state expresses his belief 
that the state needs to mandate that colleges track Tech Prep students. 



College presidents ask me where all the Tech Prep students are. I respond that 
they should have a system to identify students with articulated Tech Prep credit 
on their high school transcripts. I feel that TEA and THECB should mandate that 
colleges track students with articulated Tech Prep credit. 



This same respondent goes on to discuss the challenges in documenting student 
outcomes in his region. 



The major barrier is that schools are not motivated to do the extra work in 
documenting student outcomes. Large urban schools do not have time to 
document student competencies. We have champions in about a third of the 
schools, who believe in assessment. Many of the schools will not release data 
with social security numbers because of concerns with student's privacy. 

Schools are overwhelmed with the number of Tech Prep students, and only 
address requests for data that are tied to funding. Statewide PEIMS data is 
inaccurate. Having career majors in PEIMS data would be very helpful. This is a 
political hot potato and Tech Prep can not get the state to mandate this. We are 
trying to use a SCANTRON system where students fill out a form with personal 
information, courses they have taken, and their plans for college. This form only 
gets filled out if students are taking a course for articulated credit. The present 
survey system that they use to gather this data works in rural schools, where the 
regional Tech Prep director can work to get surveys filled out. However, in the 
urban area, only a small number of schools come to the meetings to find out how 
to fill out the survey. 



Each state agency presently tracks students enrolled in either the secondary system, 
the postsecondary system, or in the workforce independently of each other. A 



coordinated system between these three agencies that could track students across 
sectors would make documenting Tech Prep effectiveness much easier. With such a 
system, Tech Prep students coming out of any high school in the state could be tracked 
to see what percentage successfully completed a postsecondary Tech Prep program, 
and/or successfully gained a job in the field for which they were trained. An agency like 
the Texas Higher Education Coordinating could serve as the lead agency in maintaining 
this longitudinal student tracking system. They could issue reports to each of the Tech 
Prep consortia within the state with data on the graduation, transfer, and placement 
rates of all Tech Prep students from that consortium. The provision of such a 
longitudinal student tracking system by the state would lead to uniformity in the 
reporting of a number of Tech Prep student outcomes, and would provide the state with 
accountability measures for its investment in workforce education. 

This request for statewide longitudinal tracking system would not obviate the 
need for individual Tech Prep programs to create their own longitudinal student tracking 
systems to document program specific student outcomes. These program-specific 
student outcomes might include academic outcomes, like the passage rate on a state 
licensure exam or other comprehensive exam, as well as measures such as student 
satisfaction with preparation for further education or work. The purpose of individual 
Tech Prep programs maintaining a longitudinal student tracking system would be to 
determine what experiences in their program have a positive influence on the 
development of particular student outcomes. With this information, stakeholders in 
Tech Prep programs would know how they might change their policies and practices to 
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maximize the development of a particular student talent. Several phone interview 
respondents expressed this need to collect local data. 



There are big problems in trying to track students at the state level. Student 
outcomes are better tracked at the local level, where the local community and 
business have more control over the outcomes to be assessed. At the local 
level, consortia have a better feel for the types of competencies that businesses 
are seeking. It is easy to begin assessing outcomes at the local level. We are 
developing a software program to enter information about student upon entering 
their program. 



Another common theme, expressed by a number of Tech Prep directors, is that both 
the state and the local consortia need to be involved in documenting student 
performance. 



Both the local level and the state need to share responsibility for documenting 
student outcomes. We do not yet have a system in place to track students, but 
we would like to know how many Tech Prep students are matriculating in college. 



A third respondent discusses the issue of focusing on employer satisfaction. 



The fundamental question is whether employers are satisfied with the 
preparation of Tech Prep students. We are tracking students in our Tech Prep 
programs, using a computerized system that is consortium-wide. We talk to 
employers, and use follow-up questionnaires. 



Another respondent gives a more detailed description of the student tracking system 
that they utilize. 
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We use an Access database to keep track of Tech Prep student grades, and to 
run comparisons of GPA with non-Tech Prep students. We gather information 
on ethnicity, age, courses, grades, and run reports on GPA for each program. 
The Tech Prep directors should be responsible for deciding outcomes, and the 
local consortia should be responsible for gathering outcomes data on their own 
students. 



Yet another Tech Prep director discusses the actions taken by her consortium in 
developing a local student tracking system. 



We have developed a database of 200 students that came from high school who 
enrolled in college, with information on what program they came from in high 
school, how many credit hours they received. Keeping track of student 
outcomes gives credibility to Tech Prep programs. 



Another respondents discusses the challenges of creating their own student tracking 
system. 



One of the biggest challenges is the issue of time. Tech Prep directors have a 
limited amount of time and energy. In particular, cooperation between sectors is 
not always easy to develop. One of the challenges is that the PEIMS data that is 
presently collected does not give a true picture of Tech Prep effectiveness, and 
is often somewhat out-of-date. We have a student tracking system that tracks 
students through a four-year internship program. We find out if students obtain a 
job after they graduate. Surveys are sent out to ISDs, and the reports are 
broken down by grade as to how many Tech Prep students they have. 
Unfortunately, we get a low response rate, which calls into question the credibility 
of the data. 



Another Tech Prep director also discusses the challenges of gathering data on Tech 
Prep student outcomes: 



One of the biggest challenges is the structure of the consortium. The consortium 
staff are facilitators, and have little control over the colleges in the consortium. 
High schools and community colleges in Texas pride themselves on being 
independent. I feel that one of the biggest challenges is that student outcomes 
have not been defined in a consistent manner by the state, nor is there 
consensus on how to gather the data. Right now, all that is available is TAAS 
scores, student achievement test data, and GPA. 



A Tech Prep director in a major urban area expresses his belief that local consortia 
must be involved in documenting student outcomes: 



I feel that documenting student outcomes is critical, to ensure that parents and 
employers know what students have accomplished. It is also important for 
students to be aware of their own progress. I believe that assessment 
information is essential to improve programs, and to document that efforts 
devoted to Tech Prep programs are worthwhile, that there is a positive return. I 
feel that empirical data is essential to make the case for Tech Prep funding after 
federal funding is no longer available. I feel that gathering local data on student 
outcomes is critical, but I feel ineffective in gathering data. 



Clearly, Tech Prep student performance is being assessed in a variety of ways at 
different levels within the state of Texas. These combined efforts are a good start to 
documenting student outcomes in Tech Prep programs. However, as has been 
discussed in this report, additional steps need to be taken to further strengthen present 
efforts to document Tech Prep program effectiveness. The following 
recommendations, based on the data gathered from the 1996-97 are made as 
suggestions for improving efforts at Tech Prep student performance assessment: 



• Assess Tech Prep student performance at the level of the 
program, measuring both cognitive and affective outcomes. 

• Create longitudinal student tracking systems to track program- 
specific outcomes, such as student success rate on licensure 
exams, and student satisfaction. 

• Create a statewide longitudinal tracking system to track 
consistently such Tech Prep student outcomes as graduation, 
transfer, and job placement rates. 

• Involve business and industry experts in verifying mastery of 
employability and job competency skills in work-based 
components of Tech Prep programs. 

Conclusion 

Tech Prep programs have had a positive impact on Tech Prep students' preparation for 
further education and work. Documenting this positive impact is presently being 
conducted primarily at the level of the course, where Tech Prep educators have 
established exams, competency profiles, and portfolios to verify that students have met 
school- and work-based competencies. To strengthen the efforts to document the 
effectiveness of Tech Prep programs, additional efforts need to be made to strengthen 
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performance assessment at the level of the program, to further involve business and 
industry experts in verifying job and employability competencies, and to come up with a 
uniform statewide method of tracking student graduation, transfer, and job placement 
rates. These additional efforts to document Tech Prep student outcomes would go a 
long way towards demonstrating that Tech Prep educational reform has significantly 
improved the workforce education system in the state of Texas. 
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1 996-97 Project Year 

Advisory Council Members 



Mr. Robert Franks 
Director, Tech Prep at Navarro 
Navarro College 
3200 West 7th Avenue 
Corsicana, TX 75110 
(903) 874-6501 



Ms. D’Arcy Poulson 

Division Director of Workforce Education 

Howard College 

3197 Executive Drive 

San Angelo, TX 76904 

(915) 947-9572 



Dr. Lee W. Sloan 

Dean, Division of Occupational Education 
and Technology 
Del Mar College, West Campus 
Corpus Christi, TX 78404-3897 
(512) 886-1200 



Appendix B 

Minutes of Advisory Council Meeting 



MINUTES 

ADVISORY COUNCIL MEETING 

March 26, 1997 
Austin, Texas 



Project Number: 
Project Title: 

Project Year: 



77130001 

Effecive Tech Prep Policies and Practices: Performance 
Assessment 

July 1, 1996 through June 30, 1997 



[This meeting with the Advisory Council Members is in lieu of the meeting that 
had been planned for May 1, 1997, in Lubbock.] 



Attendees : 

Dr. Lee W. Sloan (Advisory Council Member) 

Dean, Division of Occupational Education and Technology 
Del Mar College, West Campus 
Corpus Christ! 

(512) 886-1200 

Mr.Robert Franks (Advisory Council Member) 

Director, Tech Prep at Navarro College 

Navarro College 

3200 West 7th Avenue 

Corsicana 

(903) 874-6501 

Dr. Ronald Opp (SPECAP) 

Project Director 
Texas Tech University 
Lubbock 

(806) 742-1997 x 294 

Dr. Oliver Hensley (SPECAP) 

Principal Investigator 
Texas Tech University 
Lubbock 

(806) 742-1997 x 281 



Mr. Clifford Chambers (SPECAP) 

Research Assistant 
Texas Tech University 
Lubbock 

(806) 742-1997 x 323 

Ms. Stacy Garrett (SPECAP) 

Research Assistant 
Texas Tech University 
Lubbock 

(806) 742-1997 x 322 

Ms. Gloria Stewart (SPECAP) 

Research Assistant 
Texas Tech University 
Lubbock 

(806) 742-1997 x 321 
Not Attending : 

Ms. D’Arcy Poulson (Advisory Council Member) 

Director, Workforce Education 

Howard College 

San Angelo 

(915) 947-9578 



Items of Discussion 



I. Document Analysis 

Advisory Council members had been asked for feedback on the criteria that 
should be used to evaluate documents for inclusion into the handbook. Based 
on their feedback, revisions were made to the form to be used to evaluate the 
documents. Three SPECAP staff members will rate each document 
independently and anonymously. The scores of the three reviewers will then 
be averaged. Documents with the highest ratings will be included in the 
handbook. 



II. Surveys 

Advisory members had provided input regarding the survey format. Changes 
were made to the survey based on thier feedback. One thousand surveys were 
distributed to all 25 Texas Tech Prep consortia. Each consortium received a 
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proportion of surveys based on their student Tech Prep enrollment. Directors 
of consortia were asked to distribute the surveys to stakeholders within their 
region who were most knowledgeable about the assessment processes in 
use. The data from surveys that have been received have been entered for 
statistical analysis. All usable surveys received through April 11th will be 
included in the analysis. Thus far, approximately 10% of the surveys have been 
returned. Thus far, approximately 15 surveys that have been returned are 
unusable, e.g., incomplete data. 



III. Telephone Interviews 

The telephone protocol was revised based on feedback from Advisory Council 
Members. We attempted to schedule telephone interviews with a staff member 
from every Tech Prep consoritum. We asked Tech Prep staff to identify one 
other individual within their consortium whom we could contact to interview 
regarding performance assessment. We have completed those interviews 
(30) and are in the process of analyzing the data. 



IV. Project Products 

The product for this project year will be a handbook containing findings from the 
document analysis, surveys, and telephone interviews. We will be asking the 
Advisory Members to review a copy of the product before it is printed. In 
addition, before publication and distribution, a draft will be sent to Dr. Brown 
and Mr. Laird for approval. Changes will be made per feedback from Advisory 
Council members and project monitors. It is anticapated that the handbook will 
be completed in May and distributed in June to all Tech Prep consortia. 



V. Presentation at the State Conference 

We will be presenting our findings from the 1995-96 project year at the state 
conference on Wednesday afternoon. 



Appendix C 

Phone Interview Protocol 
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1997 TECH PREP STUDENT OUTCOMES ASSESSMENT 
INTERVIEW PROTOCOL 
FOR 

Tech Prep Consortium Staff 



Consortium Affiliation 



□□ Alamo 
dl Brazos Valley 
d Capital 
d Central Texas 
d Coastal Bend 
d Concho Valley 
d Deep East Texas 
d East Texas 
d Global Edge 
d Golden Crescent 
d Gulf Coast 
d Heart of Texas 
d Lower Rio Grande Valley 



I — I North Central Texas 
I — I North Texas 
I — I Panhandle 
I — I Permian Basin 
I — I Southeast Texas 
I — I South Plains 
I — I South Texas 
I — I Star Tech Prep 
I — I Texoma 
I — I Upper East Texas 
I — I Upper Rio Grande Valley 
I — I West Central Texas 



Interview was conducted on 



Name 

Title 

Organization 

Address 

City 

Zip Code 

Telephone Number 
FAX Number 



with: 

(Date) 



The impact of Tech Prep 

1. How is documenting student outcomes assessment essential to the Tech Prep 
movement/philosophy? 

2. How have Tech Prep programs impacted student preparation for continuous 
lifelong learning? 

3. How have Tech Prep programs impacted student preparation for work? 

The Assessment Process 

4. With what stages of the student outcomes assessment process has your 
consortium been involved? 

cm Development 
cm Implementation 
cm Evaluation 
cm Improvement 

5. What role(s) did your consortium play in the process? (For example, facilitator, 
coordinator of efforts among stakeholders, resource provider, monitor, etc.) 

6. What were some of the greatest challenges your consortium encountered in 
developing, implementing, evaluating, and improving the student outcome 
assessment plans? 

7. In developing the plans, what attempts were made to integrate the plans along a 
continuum going from secondary education, to community colleges, to senior 
institutions, and to the work place? 

Development & implementation 

8. Please describe to me some of the best development and implementation student 
outcome assessment practices being used in your consortium. 

Evaluation & improvement 

9. Please describe to me some of the best evaluation and improvement student 
outcome assessment practices being used in your consortium. 

Additional information 

10 Would you like to add anything about assessment of student outcomes in your 
region that we may not have covered or expound on a particular topic? Or, is there 
anything you would like to ask me? 



1997 TECH PREP STUDENT OUTCOMES ASSESSMENT 
INTERVIEW PROTOCOL 
for 

Stakeholders 



Career Pathway 

□ Agriculture 
l=l Allied Health 
□□ Business 
□□ Computer Systems 

Program Name 

Consortium Affiliation 



□□ Engineering Technology 
□□ Industrial/Trade 
□□ Protection Services 
l=l Other 



□ Alamo 

im Brazos Valley 

□□ Capital 

□□ Central Texas 

□□ Coastal Bend 

□□ Concho Valley 

□□ Deep East Texas 

□□ East Texas 

CD Global Edge 

□□ Golden Crescent 

□□ Gulf Coast 

CD Heart of Texas 

CD Lower Rio Grande Valley 



I — I North Central Texas 
I — I North Texas 
I — I Panhandle 
I — I Permian Basin 
I — I Southeast Texas 
I — I South Plains 
I — I South Texas 
I — I Star Tech Prep 
I — I Texoma 
I — I Upper East Texas 
I — I Upper Rio Grande Valley 
I — I West Central Texas 



Interview on 

Name 



(Date) 



with: 



Title 

Organization 

Address 

City 

Zip Code 

Telephone Number 
FAX Number 



The Student Outcome Assessment Stakeholder Group 

1. Prior to your participation in the student outcome assessment planning process, 
have you previously served on a committee, attended a professional development 
workshop, or enrolled in a course on student outcome assessment? 

a No. 

Yes. Can you tell me how they helped prepare you for your role in 
the student outcome performance process? 

2. With what stages of the assessment of student outcomes plan have you been 
involved? 



□ Development □ Implemenation □Evaluation □Improvement 

3. Please describe the composition of the group and the role/expertise each member 
brought/provided. 

The Assessment Process 

4. Was the student outcome assessment plan 

□ an entirely new plan or aa modification of an existing plan 

5. How long did it take you to develop, implement, and evaluate your student 
outcomes assessment plan? 

6. In developing the plan(s), what attempts were made to integrate the plans along a 
continuum going from secondary education, to community colleges, to senior 
institutions, to the work place? 

Development and Implementation 

7. What were some of the challenges you encountered and the efforts to address 
these challenges in the development and implementation of the student outcome 
assessment plan, e.g., not enough time, antiquated computer system, legal 
restrictions, etc.? 

Evaluation & Improvement 

8. How often is the effectiveness of your student outcomes assessment plan 
evaluated? And, how do you go about evaluating the plan? 

9. What were some of the challenges you encountered and the efforts to address 
these challenges in the evaluation and improvement of the student outcome 
assessment plan? 

Additional Information 

10. Would you like to add anything on the assessment of student outcomes that we 
may not have covered or expound on a topic? 

59 



Appendix D 



1997 Tech Prep Student Outcome Assessment 

Questionnaire 




60 



1997 TECH PREP STUDENT OUTCOMES ASSESSMENT QUESTIONNAIRE 



Directions for Answering Questionnaire 

Your observance of these few directions will be most 
appreciated. Please focus on the assessment of student 
outcomes in the Tech Prep program with which you are 
most knowledgeable when answering the questions that 
follow. 



• Please make heavy marks that fill the oval. 



Example: Is this a survey on the assessment of student 
outcomes in Tech Prep programs? 

[ ] Yes [ ] No 



Part I - Demographic Characteristics 



1. Please indicate your 
(Mark one only) 

[ ] Alamo 
[ ] Brazos Valley 
[ ] Capital 
[ ] Central Texas 
[ ] Coastal Bend 
[ ] Concho Valley 
[ ] Deep East Texas 
[ ] East Texas 
[ ] Global Edge 
[ ] Golden Crescent 
[ ] Gulf Coast 
[ ] Heart of Texas 
[ ] Lx>wer Rio Grande 
Valley 



consortium affiliation. 



[ ] North Central Texas 
[ ] North Texas 
[ ] Panhandle 
[ ] Permian Basin 
[ ] Southeast Texas 
[ ] South Plains 
[ ] South Texas 
[ ] Star Tech Prep 
[ ] Texoma 
[ ] Upper East Texas 
[ ] Upper Rio Grande 
Valley 

[ ] West Central Texas 



you 



2. With what Tech Prep career pathway(s) are 
presently affiliated? (Mark all that apply) 



[ ] Agriculture 
[ ] Allied Health 
[ ] Business 
[ ] Computer Systems 
[ ] Engineering Technology 
[ ] Industrial/Trade 
[ ] Protection Services 
[ ] Human Services 
[ ] Other 

3. Which stakeholder group(s) do you represent? 
(Mark all that apply) 

[ ] High school faculty 
[ ] High school administrator 
[ ] Community college faculty 
[ ] Community college administrator 
[ ] Four-year college faculty 
[ ] Four-year college administrator 
[ ] Business/industry representative 
[ ] Labor representative 
[ ] Government representative 
(PIC,WDB, etc.) 

[ ] Other 



Part n - Involvement in Student 
Outcomes Assessment 

4. Have you participated in any of the following 
student outcomes assessment activities? 

(Mark all that apply) 

[ ] Served on a student outcomes 
assessment committee 
[ ] Attended a professional development 
workshop on student outcomes assessment 
[ ] Enrolled in a course on assessing student 
outcomes 
[ ] Other 

5. With what stage(s) of the assessment of student 
outcomes have you been involved? 

(Mark all that apply) 

[ ] Development of the assessment plan 
[ ] Implementation of the assessment plan 
[ ] Evaluation of the assessment plan 
[ ] Improvement of the assessment plan 

6. What role(s) have you played in developing your 
student outcomes assessment plan? 

(Mark all that apply) 

[ ] Academic subject matter expert 
[ ] Student outcomes assessment expert 
[ ] Vocational/technical subject matter expert 
[ ] Workplace competencies expert 
[ ] Leader 

[ ] Resource provider 
[ ] Other 



Part m - The Assessment Process 

7. How long did it take you to develop, implement, 
and evaluate your student outcomes assessment 
plan? (Please indicate the number of months on the 
line below) 

months 

8. What percentage of your total time did it take to 
develop, implement, and evaluate your 
student outcomes assessment plan? 

(Please indicate the percentage on the line below) 

% 



9. What groups were involved in developing, 
implementing, and evaluating your student 
assessment plan? (Mark all that apply) 

[ ] High school faculty 

[ ] High school administrators 

[ ] Community college faculty 

[ ] Community college administrators 

[ ] Four-year college faculty 

[ ] Four-year college administrators 

[ ] Business/industry representatives 

[ ] Labor representatives 

[ ] Government representatives (PIC, WDB, etc) 

[ ] Other 



PLEASE TURN OVER 



10. Please indicate your agreement with each of the 
following statements about your assessment 
planning process. (Circle one for each statement) 

SA = Strongly agree 
A = Agree 
D = Disagree 
SD = Strongly disagree 

Adequate time and resources 

were provided SA......A.........D.........SD 



Part V - Evaluating/Improving the Student 
Outcomes Assessment Plan 

15. How often do you evaluate the effectiveness of your 
student outcomes assessment plan? 

(Mark one only) 

[ ] As needed 
[ ] Every year 
[ ] Every two years 
[ ] Every three years 



Adequate staff development 

was provided SA......A.........D.........SD 

All essential stakeholder 

groups were involved SA......A.........D.........SD 

Part IV - Developing/Implementing the 
Student Assessment Plan 

11. Was your student outcomes assessment plan: 
(Mark one only) 

[ ] created as an entirely new plan? 

[ ] modified from an existing plan? 

12. Your student assessment plan incorporates 
outcomes at what levels? 

(Mark all that apply) 



16. Please indicate your agreement with each of the 
following statements about your student 
assessment plan. (Circle one for each statement) 

SA = Strongly agree 
A = Agree 
D = Disagree 
SD = Strongly disagree 

Our plan captures the 
multiple talents we are 

developing SA...... A ........ D ........ SD 

Our plan provides 
meaningful feedback 

to faculty SA...... A ........ D ........ SD 

Our plan provides 
meaningful feedback 

to students SA...... A ........ D ........ SD 



[ ] High school 
[ ] Community college 
[ ] Four-year college 
[ ] Employer follow-up 



Our plan provides 
meaningful feedback 

to employers S A...... A ........ D ........ SD 



13. What types of student outcomes assessment do you 
presently utilize? (Mark all that apply) 

[ ] Academic subject matter competency assessment 
[ ] Job competency assessment 
[ ] SCANS competency assessment 
[ ] Vocational/technicd subject matter competency 
assessment 

[ ] State licensure examination 
[ ] Other 

14. What student outcomes assessment modalities do 
you presently utilize? (Mark all that apply) 

[ ] End-of-course examination 
[ ] End-of-program examination 
[ ] Job competency examination 
[ ] Job placement tracking system 
[ ] Student tracking system 
[ ] Student competency profile 
[ ] Student portfolio 
[ ] Other 



17. Please indicate your agreement with each of the 
following statements about student outcomes 
assessment. (Mark one for each statement) 

SA = Strongly agree 
A = Agree 
D = Disagree 
SD = Strongly disagree 

Documenting student 
outcomes is essential 

to Tech Prep S A...... A ........ D ........ SD 

Tech Prep programs have 
had a positive impact on 
student preparation for 

further education S A...... A ........ D ........ SD 

Tech Prep programs have 
had a positive impact on 
student preparation 

for work SA...... A ........ D ........ SD 



Please return your completed questionnaire in the postage-paid envelope to: 



Texas Tech University 
Business Reply Center 
Box 45017 

Lubbock TX 79409-9989 



THANK YOU FOR YOUR PARTICIPATION! 
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TEXAS TECH UNIVERSITY 



SPECAP 



Box 41071, Lubbock, TX 79409-1071 



(Strategic Planning, Curriculum Evaluation, Performance Assessment) 

Dr. Ronald Opp, Dr. Oliver Hensley 

(806) 742-2329, (806) 742-1959, FAX (806) 742-2179 



Decembers, 1996 



Dear Tech Prep Educator: 

SPECAP has the responsibility of gathering essential and exemplary documents related to Tech 
Prep planning, evaluation of the curriculum and assessment of performance. In the previous two years, 
we have gathered and analyzed documents related to planning and the curriculum. This year, we are 
focusing our efforts on identifying essential assessment documents and exemplary programs related to 
performance assessment. 

We are requesting that you and other Tech Prep stakeholders in your region consider carefully 
the existing performance assessment devices used in your region and in the State. If you can identify such 
documents, we would like for you to send SPECAP copies of these documents for inclusion in the 
SPECAP Inventory of Essential Assessment Documents. Your nominations and sample documents will 
help us to make a comprehensive document inventory and analysis.Contributors will be given an 
appropriate citation for each entry in the Handbook. 

If you would send us your essential documents by January 15,1997, SPECAP analyses and 
publication schedules will be enhanced. However, we will be grateful for your contributions at anytime. 
All entries will be refereed by our editorial board for relevance, codified, and included in the SPECAP 
Inventory. In the past, your responses have helped us produce some very popular publications, which are 
currently being shared with Tech-Prep educators around the State and nation. We are grateful for your 
past contributions and look forward to receiving your new materials. 

We believe that teachers and administrators have developed local tests and assessment programs 
that are very effective. Unfortunately, these local test and assessment programs can be identified only by 
you. Please send us a single copy of your essential assessment documents, as this is the best way to 
continue our efforts to share and disseminate information about Tech Prep in the State of Texas. To help 
you with your thinking about performance assessment documents, we are including a listing of some 
sample document topics which are currently in the SPECAP Inventory. 

All contributors of essential documents will be given a complementary copy of the Inventory. 

We expect to have the Inventory and the Handbook:Essential Documents to Promote Effective Tech Prep 
Policies and Practices available in the spring of 1997. 

Please send copies of your essential or exemplary performance assessment documents to: 



Dr. Oliver Hensley New Phone Number: (806)742-1997 ext 281 
SPECAP New Phone Number: 806 742-1997 exts 321,322, or 323 
Texas Tech University 



MS 1071 

Lubbock, TX 79409 




12-3-96/Handbook/SPECAP#l 

An EEO /Affirmative Action Institution 
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Appendix F 

Document Analysis Rating System 



Document Title 



Document Rating: 

® Rater 

(rate from 1 [lowest] to 20 [highest]) 

1 . Relevance to Student Perfonnance Assessment | 

2. Relevance to Tech Prep in Texas I 



(rate from 1 [lowest] to 10 [highest]) 

3. Clarity 

4. Usefulness to Practitioners 

5. Transferability 

6. Simplicity of Data 

7. Credibility/Source of Data 

8. Quality Standards 

9. National Importance 

10. Should a sample of the document be included 




Comments: 




'.O 

>o 



Enrollment Totals By Consortia 

Per Status Report. April 1996 : Region V Education Service Center 



Consortia Name 


Calculation 


Actually 

Mailed 


Enrollment 

Secondary 


Enrollment 
Post Second! 


Alamo 


3% 


30 


4,468 


475 


Brazos Valley 


1 % 


10 


1,934 


201 


Capital 


5% 


50 


5,994 


1,785 


Central Texas 


3% 


30 


2,194 


2,718 


Coastal Bend 


4% 


40 


3,775 


1,886 


Concho Valley 


2% 


20 


2,881 


739 


Deep East 


1 % 


10 


1,297 


652 


East Texas 


2% 


20 


2,020 


536 


Global Edge 


6% 


60 


4,471 


4,071 


Golden 


4% 


40 


4,000 


1,750 


Gulf Coast 


10% 


100 


12,000 


2,500 


Heart of Texas 


1 % 


10 


912 


606 


Lower Rio 


2% 


20 


3,000 


828 


North Central Texas 


25% 


250 


23,081 


14,489 


North Texas 


1 % 


10 


1,109 


432 


Panhandle 


5% 


50 


6,000 


2,000 


Permian Basin 


2% 


20 


1,379 


1,496 


South Plains 


2% 


20 


1,500 


700 


South Texas 


1 % 


10 


208 


120 


Southeast Texas 


1 % 


10 


1,005 


607 


Star 


1 % 


10 


890 


54 


Texoma 


3% 


30 


1,537 


2,209 


Upper East 


1 % 


10 


1,200 


77 


Upper Rio 


12% 


120 


16,766 


1,795 


West Central Texas 


3% 


30 


4,000 


500 



ERIC'otals 



1,000 ,r 



100 % 



107,621 



43,098 
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February 1997 

Ms. Sylvia Kelley 
Director 

Tech Prep Global Edge 
2200 West University Drive 
McKinney, TX 75070 



Dear Ms. Kelley: 

This letter is to inform you in advance that we will be asking for your help in distributing 
a questionnaire on student performance assessment to the experts in your consortium 
who have knowledge of student performance assessment activities. We have greatly 
simplified this year's questionnaire to make it easier for the respondents in your 
consortium to answer. 

The questionnaire is presently being printed, and we hope to have it mailed to you 
within the next two weeks. The number of questionnaires you receive for distribution will 
be proportionally based on the number of high school and community college students 
our records show your consortium has enrolled in Tech Prep programs. Your 
consortium will be receiving 40 questionaires. 

We thank you in advance for your continuing support of our research endeavors to 
identify and disseminate information on exemplary Tech Prep policies and practices in 
Texas. 

Sincerely, 



Dr. Ronald D. Opp 
Project Director 



Dr. Oliver D. Hensley 
Principal Investigator 
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IMPACIWK Impact Wbrk 



Valid 



Value Label 


Value 


Frecguency 


Percent 


Percent 


Disagree 


2 


6 


5.3 


5.6 


Agree 


3 


41 


36.3 


38.3 


Strongly Agree 


4 


60 


53.1 


56.1 




0 


6 


5.3 


Missing 




Total 


113 


100.0 


100.0 



Cum 

Percent 

5.6 

43.9 

100.0 



Valid cases 



107 



Missing cases 



6 



IMEACIED Irtpact Bducaticsn 




Valid Cum 



Value Label 


Value 


Frequency 


Percent 


Percent 


Percent 


Disagree 


2 


6 


5.3 


5.6 


5.6 


Agree 


3 


44 


38.9 


40.7 


46.3 


Strongly Agree 


4 


58 


51.3 


53.7 


100.0 




0 


5 


4.4 


Missir^ 






Total 


113 


100.0 


100.0 





Valid cases 108 Missing cases 5 




72 



FEEEFAC Feedback to Facutly 



Value Label Value 

Strongly Disagree 1 

Disagree 2 

Agree 3 

Strongly Agree 4 

0 



Total 







Valid 


Qm 


Frequency 


Percent 


Percent 


Percent 


6 


5.3 


6.3 


6.3 


10 


8.8 


10.5 


16.8 


67 


59.3 


70.5 


87.4 


12 


10.6 


12.6 


100.0 


18 


15.9 


Missing 




113 


100.0 


100.0 





Valid cases 



95 



Missing cases 



18 



FEEE3SID Feec3back to Students 



Value Label Value 

Strongly Disagree 1 

Disagree 2 

Agree 3 

Strongly Agree 4 

0 



Total 



Frecguency 


Percent 


Valid 

Percent 


Cum 

Percent 


4 


3.5 


4.2 


4.2 


18 


15.9 


18.8 


22.9 


60 


53.1 


62.5 


85.4 


14 


12.4 


14.6 


100.0 


17 


15.0 


Missing 




113 


100.0 


100.0 





Valid cases 



96 



Missing cases 



17 



MJUITIMIj Multiple Talents 











Valid 


Cum 


Value Label 


Value 


Frequency 


Percent 


Percent 


Percent 


Strongly Disagree 


1 


4 


3.5 


9 

4.2 


4.2 


Disagree 


2 


12 


10.6 


12.5 


16.7 


Agree 


3 


71 


62.8 


74.0 


90.6 


Strongly Agree 


4 


9 


8.0 


9.4 


100.0 




0 


17 


15.0 


Missing 






Total 


113 


100.0 


100.0 





Valid cases 



96 



Missing cases 



17 



FEEDER Feedback Biplqyer 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Strongly Disagree 


1 


5 


4.4 


5.4 


Disagree 


2 


27 


23.9 


29.0 


Agree 


3 


51 


45.1 


54.8 


Strongly Agree 


4 


10 


8.8 


10.8 




0 


20 


17.7 


Missing 




Total 


113 


100.0 


100.0 



Valid cases 93 Missing cases 20 




Cum 

Percent 

5.4 

34.4 

89.2 

100.0 
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ACTCOyiM A^tivity-Ccmnittee 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


80 


69.6 


69.6 


69.6 


Iferked 


1 


35 


30.4 


30.4 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 115 Missing cases 0 



ACICOUE^S i^tivity-Ccurse 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


102 


88.7 


88.7 


88.7 


lyferked 


1 


13 


11.3 


11.3 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missing cases 



0 



ACIOIHER Activity-Ot±ier 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


90 


78.3 


78.3 


78.3 


jyferked 


1 


25 


21.7 


21.7 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missing cases 



0 



ACTAZKSHP Activity-WDrkshqp 



Value Label 


Value 


Frecguency 


Percent 


Valid 

Percent 


Ctm 

Percent 


Not Lferked 


0 


43 


37.4 


37.4 


37.4 


lyferked 


1 


72 


62.6 


62.6 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missiag cases 



0 



OlFBUS Grp-Business/Iiidustry 



Valid 



Value Label 


Value 


Frequency 


Percent 


Percent 


Not lyferked 


0 


105 


91.3 


91.3 


IXferked 


1 


10 


8.7 


8.7 




Total 


115 


100.0 


100.0 



Valid cases 115 Missing cases 0 




Cum 

Percent 

91.3 

100.0 



81 



GKECCA. Grp-QC Admin. 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Not lyferked 


0 


98 


85.2 


85.2 


lyferked 


1 


17 


14.8 


14.8 




Tbtal 


115 


100.0 


100.0 



Valid cases 



115 



Missing cases 



0 




Cum 

Percent 

85.2 

100.0 



82 



GRPGCF Q:p-QC Facajlty 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


99 


86.1 


86.1 


86.1 


lyferked 


1 


16 


13.9 


13.9 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 115 Missing cases 0 



dlEFYTOd Grp-FY Admin. 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


115 


100.0 


100.0 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missing cases 



0 



GKFFYEAC Grp-FY Faculty 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not l^^ked 


0 


115 


100.0 


100.0 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 115 Missing cases 0 



GKPGVT GEi:p-Govt. 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


112 


97.4 


97.4 


97.4 


lyferked 


1 


3 


2.6 


2.6 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missing cases 



0 



GREHSACM Grp-HS Acitun. 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Net lyferked 


0 


68 


59.1 


59.1 


59.1 


IXferked 


1 


47 


40.9 


40.9 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missing cases 



0 



GKEHSF Grp-HS Faculty 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


70 


60.9 


60.9 


60.9 


lyferked 


1 


45 


39.1 


39.1 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 115 Missing cases 0 




88 



ORELBR Gtp-LabDr 



Valid 



Cum 



Value Label 


Value 


Frequency 


Percent 


Percent 


Percent 


Not Iferked 


0 


115 


100.0 


100.0 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 115 



Missing cases 



0 




89 



GKEOIHER Grp-Otiier 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


106 


92.2 


92.2 


92.2 


lyferked 


1 


9 


7.8 


7.8 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 115 Missing cases 0 



EAIHAGRI Pal±imy-J^iculture 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


83 


72.2 


72.2 


72.2 


lyferked 


1 


32 


27.8 


27.8 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missing cases 



0 



PMHALI BathsflHy-Allied Health 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not Mbrked 


0 


78 


67.8 


67.8 


67.8 


lyferked 


1 


37 


32.2 


32.2 


100.0 




Idtal 


115 


100.0 


100.0 





Valid cases 115 Missing cases 0 




32 



PAIHBUS Pat±&\ay-Business 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not Marked 


0 


37 


32.2 


32.2 


32.2 


M^ked 


1 


78 


67.8 


67.8 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 115 Missiag cases 0 




93 



EAIHOCMP E^t±iway-Carpiter Systeris 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not IXferked 


0 


73 


63.5 


63.5 


63.5 


jyferked 


1 


42 


36.5 


36.5 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 115 Missiag cases 0 



PMHEISIG E^thifl^y-ESngineerijig Tech 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


83 


72.2 


72.2 


72.2 


jyferked 


1 


32 


27.8 


27.8 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missicg cases 



0 



PAIHnsiOT E^t±E/\ay-Industrial/T^ 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


69 


60.0 


60.0 


60.0 


lyferked 


1 


46 


40.0 


40.0 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missing cases 



0 



PIHBiMNSV I^thmy-Bjman Svcs . 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


79 


68.7 


68.7 


68.7 


lyferked 


1 


36 


31.3 


31.3 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missing cases 



0 



PIHGO? Pat±imy-Other 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


96 


83.5 


83.5 


83.5 


lyferked 


1 


19 


16.5 


16.5 


100.0 




Ibtal 


115 


100.0 


100.0 





Valid cases 115 Missing cases 0 




98 



PIHFRIcr Eat±iway-Protectian Svcs . 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Net lyferked 


0 


92 


80.0 


80.0 


80.0 


lyferked 


1 


23 


20.0 


20.0 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 115 Missing cases 0 



KXEACAD Role-Acac3anic E}^)ert 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Net lyferked 


0 


87 


75.7 


75.7 


75.7 


lyferked . 


1 


28 


24.3 


24.3 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missing cases 



0 



EOLELEAD Role-Leader 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not Rferked 


0 


96 


83.5 


83.5 


83.5 


lybrked 


1 


19 


16.5 


16.5 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 115 Missing cases 0 



101 



EOLEDIHR Role-Ot±ier 



Value Label 


Value 


Frequency- 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


105 


91.3 


91.3 


91.3 


lyferked 


1 


10 


8.7 


8.7 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missing cases 



0 



RODERESC Role-Resource Rrcvic3er 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


NOt M^ked 


0 


87 


75.7 


75.7 


75.7 


lyferked 


1 


28 


24.3 


24.3 


100.0 




Ttotal 


115 


100.0 


100.0 





Valid cases 



115 



Missiag cases 



0 



rOLESID Role-Std 0utcx3ne 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Qm 

Percent 


Not lyferked 


0 


99 


86.1 


86.1 


86.1 


lyferked 


1 


16 


13.9 


13.9 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 115 Missing cases 0 




104 



ROLEVOC Itole-Voc/Tech 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


55 


47.8 


47.8 


47.8 


lyferked 


1 


60 


52.2 


52.2 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 115 Missing cases 0 




105 



R0LES/O?K Role-V\for]<place 



Value Label 


Value 


Freqaency 


Percent 


Valid 

Percent 


Cum 

Percent 


Net lyferked 


0 


83 


72.2 


72.2 


72.2 


lyferked 


1 


32 


27.8 


27.8 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missing cases 



0 



SIAGEDEV Stage-Develcpnait 



Value Label 


Value 


Freqaency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


51 


44.3 


44.3 


44.3 


ly^ked 


1 


64 


55.7 


55.7 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missiag cases 



0 



STAGEE\ZA. Stage-Evaluaticn 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


70 


60.9 


60.9 


60.9 


Lferked 


1 


45 


39.1 


39.1 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missing cases 



0 



SIZ^GEIML Stage-Inplementaticn 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Q Cum 
Percent 


Not Lferked 


0 


59 


51.3 


51.3 


51.3 


lyferked 


1 


56 


48.7 


48.7 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missing cases 



0 



STAGEEMR Stege-Improvoiient 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percait 


Not lyferked 


0 


75 


65.2 


65.2 


65.2 


Marked 


1 


40 


34.8 


34.8 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missing cases 



0 



ADBQSIAF Adecguate Staff Develcpnent 











Valid 


Cum 


Value Label 


Value 


Frequency 


Percent 


Percent 


Percent 


Strongly Disagree 


1 


7 


6.2 


7.6 


7.6 


Disagree 


2 


20 


17.7 


21.7 


29.3 


Agree 


3 


49 


43.4 


53.3 


82.6 


Strongly Agree 


4 


16 


14.2 


17.4 


100.0 




0 


21 


18.6 


Missing 






Total 


113 


100.0 


100.0 





Valid cases 92 Missing cases 21 



ADBQSIMC Essential Stakdx)lders 











Valid 


Cum 


Value Label 


Value 


Frequency 


Percent 


Percent 


Percent 


Strongly Disagree 


1 


7 


6.2 


7.5 


7.5 


Disagree 


2 


19 


16.8 


20.4 


28.0 


Agree 


3 


53 


46.9 


57.0 


84.9 


Strongly Agree 


4 


14 


12.4 


15.1 


100.0 




0 


20 


17.7 


Missing 






Total 


113 


100.0 


100.0 





Valid cases 



93 



Missing cases 



20 



ADBCfTIME Adequate Time/Resources 











Valid 


Cum 


Value Label 


Value 


Frequency 


Percent 


Percent 


Percent 


Strongly Disagr^ 


1 


3 


2.7 


3.2 


3.2 


Disagree 


2 


10 


8.8 


10.6 


13.8 


Agree 


3 


62 


54.9 


66.0 


79.8 


Strongly Agree 


4 


19 


16.8 


20.2 


100.0 




0 


19 


16.8 


Missing 






Total 


113 


100.0 


100.0 





Valid cases 



94 



Missing cases 



19 



E^ZALFLfiN Evaluate Plan 




Value Label 

As Needed 
Evry Year 
Every 2 Years 
Every 3 yeairs 



Valid cases 



Value Frequency 



1 52 

2 36 

3 5 

4 1 

0 19 



Total 113 

94 Missing cases 19 





Valid 


Cum 


Percent 


Percent 


Percent 


46.0 


55.3 


55.3 


31.9 


38.3 


93.6 


4.4 


5.3 


98.9 


.9 


1.1 


100.0 



16.8 


Missing 


100.0 


100.0 





HiMOSlG Lengtii of Process 




Value label 




Valid cases 



Value Frequeacy 



1 1 

1 3 

2 7 

3 6 

4 7 

5 1 

6 8 

7 1 

8 4 

9 5 

10 5 

12 17 

14 1 

18 2 

24 8 

36 1 

0 36 



Ttotal 113 

77 Missing cases 36 



Valid Cucn 
Percent Percent Percent 



.9 


1.3 


1.3 


2.7 


3.9 


5.2 


6.2 


9.1 


14.3 


5.3 


7.8 


22.1 


6.2 


9.1 


31.2 


.9 


1.3 


32.5 


7.1 


10.4 


42.9 


.9 


1.3 


44.2 


3.5 


5.2 


49.4 


4.4 


6.5 


55.8 


4.4 


6.5 


62.3 


15.0 


22.1 


84.4 


.9 


1.3 


85.7 


1.8 


2.6 


88.3 


7.1 


10.4 


98.7 


.9 


1.3 


100.0 



31.9 


Missing 


100.0 


100.0 




J io 



INCEjQDGC Plan include CC 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


53 


46.9 


46.9 


46.9 


lyferked 


1 


60 


53.1 


53.1 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



INCLUDER Plan inclix3e enplcyer 



Value Label 


Value 


Frecguency 


Percent 


Valid 

Percent 


Cum 

Perceit 


Not lyferked 


0 


90 


79.6 


79.6 


79.6 


lyferked 


1 


23 


20.4 


20.4 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



INCLUDBY Plan incluc3e 4-year 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


104 


92.0 


92.0 


92.0 


lyferked 


1 


9 


8.0 


8.0 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



INdUEHS Plan include ES 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not M^ked 


0 


26 


23.0 


23.0 


23.0 


M^ked 


1 


87 


77.0 


77.0 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 113 Missing cases 0 





MDDBJOBC MDc3e-Job Carpetency 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Not lybrked 


0 


86 


76.1 


76.1 


Marked 


1 


27 


23.9 


23.9 




Total 


113 


100.0 


100.0 



Valid cases 



113 



Missiag cases 



0 




j 




Cum 

Percent 

76.1 

100.0 



MDDEJOBP MDde-Jcib Placetiait 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


86 


76.1 


76.1 


76.1 


lyferked 


1 


27 


23.9 


23.9 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



MDDEDIHR MDde-Ot±er 




Value Label 

Not lyferked 
lyferked 

Valid cases 113 



Value Frequency 



0 98 

1 15 

Total 113 

Missing cases 0 





Valid 


Cum 


Percent 


Percent 


Percent 


86.7 


86.7 


86.7 


13.3 


13.3 


100.0 


100.0 


100.0 






122 



lyDDEPORT MDc3e-Portfolio 



Value Label 


Value 


Freqaency 


Percent 


Valid 

Percent 


Cum 

Percent 


Nat Marked 


0 


83 


73.5 


73.5 


73.5 


Marked 


1 


30 


26.5 


26.5 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 113 Missing cases 0 





MDDEPRG MDc3e-End Prg. 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Net IXferked 


0 


79 


69.9 


69.9 


lyferked 


1 


34 


30.1 


30.1 




Total 


113 


100.0 


100.0 



Valid cases 



113 



Missing cases 



0 




Cum 

Percent 

69.9 

100.0 



124 



MDDEETOF MDde-Student Profile 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not M^ked 


0 


71 


62.8 


62.8 


62.8 


M^ked 


1 


42 


37.2 


37.2 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



MDDEIIRAC MDde-Student Tracking 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not JXferked 


0 


87 


77.0 


77.0 


77.0 


Iferked 


1 


26 


23.0 


23.0 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 113 Missing cases 0 



MDDEIGOGR MDde-ES:]d-Course 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Nat Lferked 


0 


47 


41.6 


41.6 


41.6 


Iferked 


1 


66 


58.4 


58.4 


100.0 




Idtal 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



EOJsr 



Vfes plan 




Value Label 

Entirely Nsv Plan 
Modified Plan 

Valid cases 95 



Value Frequency 



1 39 

2 56 

0 18 

Total 113 

Missing cases 18 





Valid 


Cum 


Percent 


Percent 


Percent 


34.5 


41.1 


41.1 


49.6 


58.9 


100.0 


15.9 


Missing 




100.0 


100.0 







PROCBUS Pmcess-Busiiiess/ Indi LStry 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


CXm 

Percent 


Not Iferked 


0 


65 


57.5 


57.5 


57.5 


Iferked 


1 


48 


42.5 


42.5 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 113 Missing cases 0 




129 



PE^QXCMD Process-QC Admin. 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not Lferked 


0 


70 


61.9 


61.9 


61.9 


Lferked 


1 


43 


38.1 


38.1 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



EROQCCF Process-CC Faculty 



Value Label 


Value 


Frecguency 


Percent 


Valid 

Percent 


Cum 

Percent 


Nut lyferked 


0 


54 


47.8 


47.8 


47.8 


lyferked 


1 


59 


52.2 


52.2 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missiag cases 



0 



EF?OCF!i!AD Rcocess-Four Yr Adonin 



Valid 



Cum 



Value Label 


Value 


Frequency 


Percent 


Percent 


Percent 


Not lyferked 


0 


107 


94.7 


94.7 


94.7 


lyferked 


1 


6 


5.3 


5.3 


100.0 




Tbtal 


113 


100.0 


100.0 





Valid cases 113 Missiag cases 0 




EEROCFYF Process-Four Yr Faculty 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Not lyferked 


0 


105 


92.9 


92.9 


Marked 


1 


8 


7.1 


7.1 




-Idtal 


113 


100.0 


100.0 



Valid cases 



113 



Missing cases 



0 




Cum 

Percent 

92.9 

100.0 
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EE^OOGVT PrccessHjovernment 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not M^ked 


0 


98 


86.7 


86.7 


86.7 


lyferked 


1 


15 


13.3 


13.3 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



BEOCHSF Rrocess-HS Faculty 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


31 


27.4 


27.4 


27.4 


lyferked 


1 


82 


72.6 


72.6 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



BROCHSAD Process-HS Adndn. 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Qm 

Percent 


Not lyferked 


0 


58 


51.3 


51.3 


51.3 


lyferked 


1 


55 


48.7 


48.7 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



PEOQOIHR Process-Other 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not ly^ked 


0 


94 


83.2 


83.2 


83.2 


lyferked 


1 


19 


16.8 


16.8 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



EEOZLBR Process-Labor 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Not M^ked 


0 


96 


85.0 


85.0 


lyferked 


1 


17 


15.0 


15.0 




Total 


113 


100.0 


100.0 



Valid cases 



113 



Missing cases 



0 







Cum 

Percent 

85.0 

100.0 



TIME 



Percent Total Time 



Value label 









Valid 


Cum 


Value 


Frequency 


Percent 


Percent 


Percent 


1 


2 


1.8 


2.7 


2.7 


2 


5 


4.4 


6.7 


9.3 


3 


2 


1.8 


2.7 


12.0 


5 


13 


11.5 


17.3 


29.3 


10 


18 


15.9 


24.0 


53.3 


12 


1 


.9 


1.3 


54.7 


15 


8 


7.1 


10.7 


65.3 


18 


1 


.9 


1.3 


66.7 


20 


10 


8.8 


13.3 


80.0 


22 


1 


.9 


1.3 


81.3 


25 


4 


3.5 


5.3 


86.7 


35 


1 


.9 


1.3 


88.0 


40 


1 


.9 


1.3 


89.3 


45 


1 


.9 


1.3 


90.7 


50 


2 


1.8 


2.7 


93.3 


60 


1 


.9 


1.3 


94.7 


75 


2 


1.8 


2.7 


97.3 


80 


1 


.9 


1.3 


98.7 


95 


1 


.9 


1.3 


100.0 


0 


38 


33.6 


Missing 




Total 


113 


100.0 


100.0 





Valid cases 



75 



Missing cases 



38 




1 o 

.t \j 



9 



USEACM) Use Acadsnic Subjt Matter 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


51 


45.1 


45.1 


45.1 


lyferked 


1 


62 


54.9 


54.9 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



USEJOB Use job ccirpetencies 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not jyferked 


0 


69 


61.1 


61.1 


61.1 


lyferked 


1 


44 


38.9 


38.9 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



USEDIHER Use other 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


105 


92.9 


92.9 


92.9 


lyferked 


1 


8 


7.1 


7.1 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



USESC?J^ Use SCANS 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


66 


58.4 


58.4 


58.4 


lyferked 


1 


47 


41.6 


41.6 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



USESI2^ Use State Certif Exam 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


dm 

Percent 


Net l^ferked 


0 


79 


69.9 


69.9 


69.9 


Lferked 


1 


34 


30.1 


30.1 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



USEVOC use vocational/tech 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Not lyferked 


0 


39 


34.5 


34.5 


34.5 


lyferked 


1 


74 


65.5 


65.5 


100.0 




Total 


113 


100.0 


100.0 





Valid cases 



113 



Missing cases 



0 



Appendix J 

Consortia Responding to Questionnaire 



OCNSORT Ccfnsortium 



Value Label 


Value 


Frequency 


Percent 


Valid 

Percent 


Cum 

Percent 


Brazos Vall^ 


2 


3 


2.6 


2.6 


2.6 


Coital 


3 


1 


.9 


.9 


3.5 


Central Texas 


4 


10 


8.7 


8.7 


12.2 


Ccocho Vall^ 


6 


4 


3.5 


3.5 


15.7 


De^ East 


7 


2 


1.7 


1.7 


17.4 


East Texas 


8 


5 


4.3 


4.3 


21.7 


Global Edge 


9 


8 


7.0 


7.0 


28.7 


Gulf Coast 


11 


16 


13.9 


13.9 


42.6 


Heart of Texas 


12 


3 


2.6 


2.6 


45.2 


Lcwer Rio Grande 


13 


2 


1.7 


1.7 


47.0 


North Central 


14 


15 


13.0 


13.0 


60.0 


North Texas 


15 


3 


2.6 


2.6 


62.6 


Panhandle 


16 


11 


9.6 


9.6 


72.2 


Permian Basin 


17 


6 


5.2 


5.2 


77.4 


South Texas 


20 


4 


3.5 


3.5 


80.9 


Star Tech Prep 


21 


2 


1.7 


1.7 


82.6 


Texcma 


22 


1 


.9 


.9 


83.5 


Ufeper Rio Grande 


24 


13 


11.3 


11.3 


94.8 


Vfest Central 


25 


6 


5.2 


5.2 


100.0 




Total 


115 


100.0 


100.0 





Valid cases 



115 



Missing cases 



0 
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